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Abstract — We consider multiuser scheduling in wireless net- 
works with channel variations and flow-level dynamics. Recently, 
it has been shown that the MaxWeight algorithm, which is 
throughput-optimal in networks with a fixed number of users, 
fails to achieve the maximum throughput in the presence of 
flow-level dynamics. In this paper, we propose a new algorithm, 
called workload-based scheduling with learning, which is provably 
throughput-optimal, requires no prior knowledge of channels and 
user demands, and performs significantly better than previously 
suggested algorithms. 



I. Introduction 

Multiuser scheduling is one of the core challenges in 
wireless communications. Due to channel fading and wire- 
less interference, scheduling algorithms need to dynamically 
allocate resources based on both the demands of the users and 
the channel states to maximize network throughput. The cel- 
ebrated MaxWeight algorithm developed in [3) for exploiting 
channel variations works as follows. Consider a network with 
a single base station and n users, and further assume that the 
base station can transmit to only one user in each time slot. 
The MaxWeight algorithm computes the product of the queue 
length and current channel rate for each user, and chooses 
to transmit to that user which has the largest product; ties 
can be broken arbitrarily. The throughput-optimality property 
of the MaxWeight algorithm was first established in 0, and 
the results were later extended to more general channel and 
arrival models in iHJ-Q. The MaxWeight algorithm should 
be connasted with other opportunistic scheduling such as 

0, which exploit channel variations to allocate resources 
fairly assuming continuously backlogged users, but which are 
not throughput-optimal when the users are not continuously 
backlogged. 

While the results in (|3)-||5) demonstrate the power of 
MaxWeight-based algorithms, they were obtained under the 
assumptions that the number of users in the network is fixed 
and the traffic flow generated by each user is long-lived, 

1. e., each user continually injects new bits into the network. 
However, practical networks have flow-level dynamics: users 
arrive to transmit data and leave the network after the data are 
fully transmitted. In a recent paper JT), the authors show that 
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the MaxWeight algorithm is in fact not throughput optimal 
in networks with flow-level dynamics by providing a clever 
example showing the instability of the MaxWeight scheduling. 
The intuition is as follows: if a long-lived flow does not 
receive enough service, its backlog builds up, which forces 
the MaxWeight scheduler to allocate more service to the 
flow. This interaction between user backlogs and scheduling 
guarantees the correctness of the resource allocation. However, 
if a flow has only a finite number of bits, its backlog does 
not build up over time and it is possible for the MaxWeight 
to stop serving such a flow and thus, the flow may stay 
in the network forever. Thus, in a network where finite-size 
flows continue to arrive, the number of flows in the network 
could increase to infinity. One may wonder why flow-level 
instability is important since, in real networks, base stations 
limit the number of simultaneously active flows in the network 
by rejecting new flows when the number of existing flows 
reaches a threshold. The reason is that, if a network model 
without such upper limits is unstable in the sense that the 
number of flows grows unbounded, then the corresponding 
real network with an upper limit on the number of flows will 
experience high flow blocking rates. This fact is demonstrated 
in our simulations later. 

In (TJ, the authors address this instability issue of 
MaxWeight-based algorithms, and establish necessary and 
sufficient conditions for the stability of networks with flow- 
level dynamics. The authors also propose throughput-optimal 
scheduling algorithms. However, as the authors mention in [fl~), 
the proposed algorithms require prior knowledge of channel 
distribution and traffic distribution, which is difficult and 
sometimes impossible to obtain in practical systems, and 
further, the performance of the proposed algorithms is also 
not ideal. A delay-driven MaxWeight scheduler has also been 
proposed to stabilize the network under flow-level dynamics 
0. The algorithm however works only when the maximum 
achievable rates of the flows are identical. 

Since flow arrivals and departures are common in reality, we 
are interested in developing practical scheduling algorithms 
that are throughput-optimal under flow-level dynamics. We 
consider a wireless system with a single base station and 
multiple users (flows). The network contains both long-lived 
flows, which keep injecting bits into the network, and short- 
lived flows, which have a finite number of bits to transmit. 
The main contributions of this paper include the following: 
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> We obtain the necessary conditions for flow-level stability 
of networks with both long-lived flows and short-lived 
flows. This generalizes the result in 0J, where only short- 
lived flows are considered. 

> We propose a simple algorithm for networks with short- 
lived flows only. Under this algorithm, each flow keeps 
track of the best channel condition that it has seen so far. 
Each flow whose current channel condition is equal to the 
best channel condition that it has seen during its lifetime 
is eligible for transmission. It is shown that an algorithm 
which uniformly and randomly chooses a flow from 
this set of eligible flows for transmission is throughput- 
optimal. Note that the algorithm is a purely opportunistic 
algorithm in that it selects users for transmission when 
they are in the best channel state that they have seen so 
far, without considering their backlogs. 

• Based on an optimization framework, we propose to 
use the estimated workload, the number of time slots 
required to transmit the remainder of a flow based on 
the best channel condition seen by the flow so far, to 
measure the backlog of short-lived flows. By comparing 
this short-lived flow backlog to the queue lengths and 
channel conditions of the long-lived flows, we develop a 
new algorithm, named workload-based scheduling with 
learning, which is throughput-optimal under flow-level 
dynamics. The term "learning" refers to the fact that the 
algorithm learns the best channel condition for each short- 
lived flow and attempts to transmit when the channel 
condition is the best. 

■ We use simulations to evaluate the performance of the 
proposed scheduling algorithm, and observe that the 
workload-based scheduling with learning performs signif- 
icantly better than the MaxWeight scheduling in various 
settings. 

The terminology of long-lived and short-lived flows above 
has to be interpreted carefully in practical situations. In prac- 
tice, each flow has a finite size and thus, all flows eventually 
will leave the system if they receive sufficient service. Thus, 
all flows are short-lived flows in reality. Our results suggest 
that transmitting to users who are individually in their best 
estimated channel state so far is thus, throughput optimal. On 
the other hand, it is also well known that real network traffic 
consists of many flows with only a few packets and a few flows 
with a huge number of packets. If one considers the time scales 
required to serve the small-sized flows, the large-sized flows 
will appear to be long-lived (i.e., persistent forever) in the 
terminology above. Thus, if one is interested in performance 
over short time-scales, an algorithm which considers flows 
with a very large number of packets as being long-lived may 
lead to better performance and hence, we consider the more 
general model which consists of both short-lived flows and 
long-lived flows. Our simulations later confirm the fact that the 
algorithm which treats some flows are being long-lived leads 
to better performance although throughput-optimality does not 
require such a model. In addition, long-lived flows partially 
capture the scenario where all bits from a flow do not arrive 
at the base station all at once. This fact is also exploited in 



our simulation experiments. 

II. Basic Model 

Network Model: We consider a discrete-time wireless 
downlink network with a single base station and many flows, 
each flow associates with a distinct mobile user. The base 
station can serve only one flow at a time. 

Traffic Model: The network consists of the following two 
types of flows: 

• Long-lived flows: Long-lived flows are traffic streams 
that are always in the network and continually generate 
bits to be transmitted. 

• Short-lived flows: Short-lived flows are flows that have a 
finite number of bits to transmit. A short-lived flow enters 
the network at a certain time, and leaves the system after 
all bits are transmitted. 

We assume that the set of long-lived flows is fixed, and short- 
lived flows arrive and depart. We let I be the index for long- 
lived flows, C be the set of long-lived flows, and L be the 
number of long-lived flows, i.e., L = \C\. Furthermore, we let 
Xi (i) be the number of new bits injected by long-lived flow / 
in time slot t, where Xi(t) is a discrete random variable with 
finite support, and independently and identically distributed 
(i.i.d.) across time slots. We also assume E[X;(£)] = xi and 
Xi{t) < X max for all I and t. 

Similarly, we let i be the index for short-lived flows, I(t) be 
the set of short-lived flows in the network at time t, and I(t) 
be the number of short-lived flows at time t, i.e., I(t) = \I(t)\. 
We denote by /j the size (total number of bits) of short-lived 
flow i, and assume fi < F ma - X for all i. 

It is important to note that we allow different short-lived 
flows to have different maximum link rates. A careful con- 
sideration of our proofs will show the reader that the learning 
algorithm is not necessary if all users have the same maximum 
rate and that one can simply transmit to the user with the best 
channel state if it is assumed that all users have the same 
maximum rate. However, we do not believe that this is a very 
realistic scenario since SNR variations will dictate different 
maximum rates for different users. 

Residual Size and Queue Length: For a short-lived flow 
i, let Qi(t) which we call the residual size, denote the number 
of bits still remaining in the system at time t. For a long-lived 
flow I, let Qi(t) denote the number of bits stored at the queue 
at the base station. 

Channel Model: There is a wireless link between each user 
and the base station. Denote by Ri(t) the state of the link 
between short-lived flow i and the base station at time t (i.e., 
the maximum rate at which the base station can transmit to 
short-lived flow i at time £), and Ri(t) the state of the link 
between long-lived flow I and the base station at time t. We 
assume that Ri(t) and Ri(t) are discrete random variables 
with finite support. Define i?" lax and i?j nax to be the largest 
values that these random variables can take, i.e., P(Rj(t) > 
Rf ax ) = for each j <= £{J(\J t l{t)) . Choose p™ ax > 
and i? max > such that 

Pr(ifc(t) = i?f iax ) > p^ iax Vi, t 

max {max, i?f lax , max/ i?™ ax } < i? max . 
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The states of wireless links are assumed to be independent 
across flows and time slots (but not necessarily identically 
distributed across flows). The independence assumption across 
time slots can be relaxed easily but at the cost of more 
complicated proofs. 

III. Workload-based Scheduling with Learning 

In this section, we introduce a new scheduling algorithm 
called Workload-based Scheduling with Learning (WSL). 
Workload-based Scheduling with Learning: For a short- 
lived flow i, we define 



max Ri(s), 

max{t-D,b;}<s<t 



where bi is the time short-lived flow i joins the network and 
D > is called the learning period. A key component of this 
algorithm is to use R™ ax to evaluate the workload of short- 
lived flows (the reason will be explained in a detail in Section 
fVl l. However, i?f lax is in general unknown, so the scheduling 
algorithm uses i?™ ax (t) as an estimate of i?f lax . 

During each time slot, the base station first checks the 
following inequality: 



E 

iei(t) 



Qi(t) 

R^it) 



> max( 



i(t)Ri(t), 



(1) 



where a > 0. 

• If inequality (Q~|i holds, then the base station serves a short- 
lived flow as follows: if at least one short-lived flow (say 
flow i) satisfies R t {t) > Q,(t) or R^t) = #f ax (i), then 
the base station selects such a flow for transmission (ties 
are broken according to a good tie-breaking rule, which is 
defined at the end of this algorithm); otherwise, the base 
station picks an arbitrary short-lived flow to serve. 

< If inequality (03 does not hold, then the base station serves 
a long-lived flow /* such that 

I* € ar g max Q i (t)Ri(t) 
(ties are broken arbitrarily). 

"Good" tie-breaking rule: Assume that the tie-breaking 
rule is applied to pick a short-lived flow every time slot 



«(*) 



> 



(but the flow is served only if a^ieift 
max; e £ Qi(t)Ri(t)). We define £ m i SS (t) to be the event that 
the tie -breaking rule selects a short-lived flow with 7?™ ax (i) ^ 
RT ax - Define 



W s (t) = 



E 

iei(t) 



R" 



which is he total workload of the system at time t. A tie- 
breaking rule is said to be good if the following condition 
holds: Consider the WSL with the given tie-breaking rule and 
learning period D. Given any e m i SS > 0, there exist N emiss 
and D Cmiss such that 

Pi - (£miss (^)) — ^raiss 

if D > D £mi3g and W s (t - D) > N €miaa . 

□ 



Remark 1: While all WSL scheduling algorithms with good 
tie-breaking rules are throughput optimal, their performances 
in terms of other metrics could be different depending upon 
the tie-breaking rules. We consider two tie-breaking rules in 
this paper: 

• Uniform Tie-breaking: Among all short-lived flows 

satisfying Ri(t) = R^{t) or R t (t) > Qi(t), the base- 
station uniformly and randomly selects one to serve. 

• Oldest-first Tie-breaking: Let denote the number of 
time slots a short-lived flow has been in the network. 
The base station keeps track = min{f, Pi} for every 
short-lived flow, where f is some fixed positive integer. 
Among all short-lived flows satisfying Ri(t) — i?f ax (i) 
or Ri{t) > Qi(t), the tie-breaking rule selects the one 
with the largest t, , and the ties are broken uniformly and 
randomly!]] 

The "goodness" of these two tie-breaking rules are proved in 
Appendix C and D, and the impact of the tie-breaking rules 
on performance is studied in Section [Vll using simulations. 

Remark 2: The a in inequality (Q~|) is a parameter balancing 
the performance of long-lived flows and short-lived flows. A 
large a will lead to a small number of short-lived flows but 
large queue-lengths of long-lived flows, and vice versa. 

Remark 3: In Theorem [3] we will prove that WSL is 
throughput optimal when D is sufficiently large. From purely 
throughput-optimality considerations, it is then natural to 
choose D = oo. However, in practical systems, if we choose 
D too large, such as oo, then it is possible that a flow may stay 
in the system for a very long time if its best channel condition 
occurs extremely rarely. Thus, it is perhaps best to choose a 
finite D to tradeoff between performance and throughput. 

Remark 4: If all flows are short-lived, then the algorithm 
simplifies as follows: If at least one short-lived flow (say flow 
i) satisfies Ri(t) > Qi(t) or R t (t) = Rf & *{t), then the base 
station selects such a flow for transmission according to a 
"good" tie-breaking rule; otherwise, the base station picks an 
arbitrary short-lived flow to serve. Simply stated, the algorithm 
serves one of the flows which can be completely transmitted 
or sees its best channel state, where the best channel state 
is an estimate based on past observations. If no such flow 
exists, any flow can be served. We do not separately prove the 
throughput optimality of this scenario since it is a special case 
of the scenario considered here. But it is useful to note that, 
in the case of short-lived flows only, the algorithm does not 
consider backlogs at all in making scheduling decisions. 

We will prove that WSL (with any a > 0) is throughput- 
optimal in the following sections, i.e., the scheduling policy 
can support any set of traffic flows that are supportable by 
any other algorithm. In the next section, we first present the 
necessary conditions for the stability, which also define the 
network throughput region. 



'We set a upper bound f on f) for technical reasons that facilitate the 
throughput-optimality proof. Since f can be arbitrarily large, we conjecture 
that this upper bound is only for analysis purpose, and not required in practical 
systems. 
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IV. Necessary Conditions for Stability 

In this section, we establish the necessary conditions for 
the stability of networks with flow-level dynamics. To get the 
necessary condition, we need to classify the short-lived flows 
into different classes. 

• A short-lived flow class is defined by a pair of random 
variables (R, F). Class-fc is associated with random vari- 
ables Rk and Ff. A short-lived flow i belongs to class k if 
Ri{t) has the same distribution as Rk and the size of flow 
i (fi) has the same distribution as Fk . We let A& (t) denote 
the number of class-fc flows joining the network at time t, 
where Ak(t) are i.i.d. across time slots and independent but 
not necessarily identical across classes, and E[Afc(t)] = A^. 
Denote by K, the set of distinct classes. We assume that K, 
is finite, \K\ = A, and A k [t] < A max for all t and k e K. 

• Let c denote an L-dimensional vector describing the state 
of the channels of the long-lived flows. In state c, R c j is 
the service rate that long-lived flow I can receive if it is 
scheduled. We denote by C the set of all possible states. 

• Let C(i) denote the state of the long-lived flows at time 
t, and 7r c denote the probability that C(t) is in state c. 

> Let pc.i be the probability that the base station serves flow 
I when the network is in state c. Clearly, for any c, we have 



lec 



Pc,i < L 



Note that the sum could be less than 1 if the base station 
schedules a short-lived flow in this state. 

• Let [x c ,s be the probability that the base station serves a 
short-lived flow when the network is in state c. 

• Let ®k,p(t) denote the number of short-lived flows that 
belong to class-fc and have residual size Q(t) = (3. Note 
that /3 can only take on a finite number of values. 
Theorem 1: Consider traffic parameters {xi} and {A^}, and 

suppose that there exists a scheduling policy guaranteeing 



lim E 

t— >QO 



lec ketc 0=1 



< oo. 



Then there exist p c j and /x c s such that the following inequal- 
ities hold: 



Xl < E^ KcPcjRcj VI e £ 
E] A fe E 



cec 



keK, 



F, 



— ^ c , s7Tc 

cec 




Mc,s < 1 Vc e C. 



(2) 
(3) 

(4) 

□ 



Inequality (f2| and ([3]) state that the service allocated should 
be no less than the user requests if the flows are supportable. 
Inequality © states that the overall time used to serve long- 
lived and short-lived flows should be no more than the time 

2 We use " to indicate that the notation is associated with a class of short- 
lived flows instead of an individual short-lived flow. 



available. To prove this theorem, it can be shown that for any 
traffic for which we cannot find p Cj / and /i c s satisfying the 
three inequalities in the theorem, a Lyapunov function can 
be constructed such that the expected drift of the Lyapunov 
function is larger than some positive constant under any 
scheduling algorithm, which implies the instability of the 
network. The complete proof is based on the Strict Separation 
Theorem and is along the lines of a similar proof in (5 ], and 
is omitted in this paper. 

V. Throughput Optimality of WSL 

First, we provide some intuition into how one can derive 
the WSL algorithm from optimization decomposition consid- 
erations. Then, we will present our main throughput optimality 
results. Given traffic parameters {xi} and {A^}, the necessary 
conditions for the supportability of the traffic is equivalent to 
the feasibility of the following constraints: 



X l - EcSC KcPc,lRc,l 



E 



keK. 



A fc E 



Ei££Pc,/ + /l c ,s < 1 



V/ 



Vc. 



(5) 



For convenience, we view the feasibility problem as an opti- 
mization problem with the objective max A, where A is some 
constant. While we have not explicitly stated that the x's and 
/Li's are non-negative, this is assumed throughout. 

Partially augmenting the objective using Lagrange multipli- 
ers, we get 



max A - J2i ec 



s.t. 



fjb c s < 1 Vc. 



For the moment, let us assume Lagrange multipliers qi and 
q s are given. Then the maximization problem above can be 
decomposed into a collection of optimization problems, one 
for each c : 



max 

Pc,(,/Jc, 



EZ«^c 



Wed + <7sMc,s 



lec 



s.t. 



E;e£Pc 



/'c 



< 1. 



It is easy to verify that one optimal solution to the optimization 
problem above is: 

* if q s > maxi e£ qiR c ,i, then fj, CiS = 1 and p c ,i = O(VZ); 

• otherwise, fj, c<s = 0, and p c .j» = 1 for some I* £ 
argmaxg/i? c .; and p c j = for other /. 

The complementary slackness conditions give 

gn xi - n c p c .iR cA = 0. 
V cec ) 

Since xi is the mean arrival rate of long-lived flow I and 
Ecec ^cPcjRcj is the mean service rate, the condition on qi 
says that if the mean arrival rate is less than the mean service 
rate, qi is equal to zero. Along with the non-negativity condi- 
tion on qi , this suggests that perhaps qi behaves likes a queue 
with these arrival and service rates. Indeed, it turns out that 



5 



the mean of the queue lengths are proportional to Lagrange 
multipliers (see the surveys in ll9l- lfTTI ). For long-lived flow I, 
we can treat the queue-length Qi (t) as a time- varying estimate 
of Lagrange multiplier qi . Similarly q s can be associated with 
a queue whose arrival rate is J2keic A , which is 

the mean rate at which workload arrives where workload is 
measured by the number of slots needed to serve a short-lived 
flow if it is served when its channel condition is the best. The 
service rate is 2~2cec Mc.sTTc which is the rate at which the 
workload can potentially decrease when a short-lived flow is 
picked for scheduling by the base station. Thus, the workload 
in the system can serve as a dynamic estimate of q s . 

Letting aW s (t) (a > 0) be an estimate of q s , the observa- 
tions above suggest the following workload-based scheduling 
algorithm if i?f lax are known. 

Workload-based Scheduling (WS): During each time slot, 
the base station checks the following inequality: 

aW a (t) > maxQi(t)Ri(t). (6) 

• If inequality (O holds, then the base station serves a short- 
lived flow as follows: if at least one short-lived flow (say 
flow i) satisfies R % {t) > Q % {t) or R^t) = i?f lax , then 
such a flow is selected for transmission (ties are broken 
arbitrarily); otherwise, the base station picks an arbitrary 
short-lived flow to serve. 

« If inequality (|6]i does not hold, then the base station serves 
a long-lived flow I* such that I* <E argmax; e £ Qi(t)Ri(i) 
(ties are broken arbitrarily). 

• The factor a can be obtained from the optimization for- 
mulation by multiplying constraint @ by a on both sides 

□ 

However, this algorithm which was directly derived from 
dual decomposition considerations is not implementable since 
i?f lax 's are unknown. So WSL uses ^™ ax (t) to approximate 
jj>max ]sj ote that an inaccurate estimate of Rf 1 ** not only 
affects the base station's decision on whether Ri(t) = i?f ax , 



However, it is not 



but also on its computation of 
difficult to see that the error in the estimate of the total 
workload is a small fraction of the total workload when the 
total workload is large: when the workload is very large, the 
total number of short-lived flows is large since their file sizes 
are bounded. Since the arrival rate of short-lived flows is also 
bounded, this further implies that the majority of short-lived 
flows must have arrived a long time ago which means that with 
high probability, their estimate of their best channel condition 
must be correct. 

Next we will prove that both WS and WSL can stabilize 
any traffic xi and A& such that (1 + e)xi and (1 + e)A/c 
are supportable, i.e., satisfying the conditions presented in 
Theorem [TJ In other words, the number of short-lived flows 
in the network and the queues for long-lived flows are all 
bounded. Even though WS is not practical, we study it first 
since the proof of its throughput optimality is easier and 
provides insight into the proof of throughput-optimality of 
WSL. 

Let 



Since the base station makes decisions on M(t) and R(t) = 
{{Ri(t)} ie x(t),{Ri(t)}i e c} under WS. It is easy to verify 
that M(t) is a finite-dimensional Markov chain under WS. 
Assume that Afc(t), Fk and Xi(i) are such that the Markov 
chain M is irreducible and aperiodic. 

Theorem 2: Given any traffic xi and Afc such that (1 + e)xi 
and (1 + e)Afc are supportable, the Markov chain M(£) is 
positive-recurrent under WS, and 



lim E 

t— >oo 



E E ■ 

iei(t) 



i(t) 



lec 



< 00. 



Proof: We consider the following Lyapunov function: 

V(t)=a(W s (t)) 2 + ^2(Q l (t)f, (7) 



lec 



and prove that 

E[V(t + 1) - V(t)\M(t)} < U d l mt)e y - ; 

+ EW^ 



[a\W s (t) 



lec 



M(t)£T 



for some Ud > 0, e > 0, A > 0, and a finite set T. Positive 
recurrence of M then follows from Foster's Criterion for 
Markov chains |fl2|, and the boundedness of the first moment 
follows from JT3|. The detailed proof is presented in Appendix 
A. 

■ 

We next study WSL, where i?f lax is estimated from the 
history. We define ®k,p,r(t) to be the number of short-lived 
flows that belong to class-fc, have a residual size of (3, and 
have R™ ax (i) = r. Furthermore, we define 



M(n) 



/ 

{Qi(t)}iecAQk,pAt)} keK 

i 1</3<_F" 
V i<r<Rt 



(n-l)T+l<t<nT 



from some T > D, It is easy to see that M(n) is a finite- 
dimensional Markov chain under WSL0 

Theorem 3: Consider traffic xi and Afc such that (1 + e)xi 
and (1 + e)Afc are supportable. Given WSL with a good tie- 
breaking rule, there exists D e such that the Markov chain 
M(n) is positive-recurrent under the WSL with learning 
period D > D e and the given tie-breaking rule. Further, 



lim E 



lec 



E W 

iei(t) 



< oo. 



M(t) = ({Q l (t)} leC ,{QkAt)} 



kelCS<P<F" 



0- 



Proof: The proof of this theorem is built upon the 
following two facts: 

> When the number of short-lived flows is large, the majority 
of short-lived flows must have been in the network for a 

3 This Markov chain is well-defined under the uniform tie-breaking rule. 
For other good tie-breaking rules, we may need to first slightly change the 
definition of M(n) to include the information required for tie-breaking, and 
then use the analysis in Appendix B to prove the positive recurrence. 
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long time and have obtained the correct estimate of the best 
channel condition, which implies that 



E 

iex(t) 



Qi(t) 



E 

iex(t) 



i?™ ax (i) 



• When the number of short-lived flows is large, the short- 
lived flow selected by the base station (say flow i) has a 
high probability to satisfy R l {t) = Rf ax or Ri(t) > Q t (t). 
From these two facts, we can prove that with a high 
probability, the scheduling decisions of WSL are the same 
as those of WS, which leads to the throughput optimality of 
WSL. The detailed proof is presented in Appendix B. 



VI. Simulations 

In this section, we use simulations to evaluate the perfor- 
mance of different variants of WSL and compare it to other 
scheduling policies. There are three types of flows used in the 
simulations: 

• S-flow: An S-fiow has a finite size, generated from a 
truncated exponential distribution with mean value 30 and 
maximum value 150. Non-integer values are rounded to 
integers. 

« M-flow: An M-fiow keeps injecting bits into the network 
for 10, 000 time slots and stops. The number of bits gen- 
erated at each time slot follows a Poisson distribution with 
mean value 1. 

< L-flow: An L-fiow keeps injecting bits into the network 
and never leaves the network. The number of bits generated 
at each time slot follows a truncated Poisson distribution 
with mean value 1 and maximum value 10. 
Here S-fiows represent short-lived flows that have finite sizes 
and whose bits arrive all at once; L-flows represent long- 
lived flows that continuously inject bits and never leave the 
network; and M-fiows represent flows of finite size but whose 
arrival rate is controlled at their sources so that they do not 
arrive instantaneously into the network. Our simulation will 
demonstrate the importance of modeling very large, but finite- 
sized flows as long-lived flows. 

We assume that the channel between each user and the base 
station is distributed according to one of the following three 
distributions: 

• G-link: A G-link has five possible link rates 
{10,20,30,40,50}, and each of the states happens with 
probability 20%. 

• P-link: A P-link has five possible link rates 
{5,10,15,20,25}, and each of the states happens 
with probability 20%. 

• R-link: An R-link has five possible link rates 
{10,20,30,40,100}, and the probabilities associated 
with these link states are {0.5,0.2,0.2,0.09,0.01}. 

The G, P and R stand for Good, Poor and Rare, respectively. 
We include these three different distributions to model the 
SNR variations among the users, where G-links represent links 
with high SNR (e.g., those users close to the base station), P- 
links represent links with low SNR (e.g., those users far away 



from the base station), and R-links represent links whose best 
state happens rarely. The R-links will be used to study the 
impact of learning period D on the network performance. 

We name the WSL with the uniform tie-breaking rule 
WSLU, and the WSL with the oldest-first tie-breaking rule 
WSLO. In the following simulations, we will first demonstrate 
that the WSLU performs significantly better than previously 
suggested algorithms, and then show that the performance 
can be further improved by choosing a good tie-breaking 
policy (e.g., WSLO). We set a to be 50 in all the following 
simulations. 



Simulation I: Short-lived Flow or Long-lived Flow? 

We first use the simulation to demonstrate the importance 
of considering a flow with a large number of packets as being 
long-lived. We consider a network consisting of multiple S- 
flows and three M-flows, where the arrival of S-fiows follows a 
truncated Poisson process with maximum value 100 and mean 
value A. All the links are assumed to be G-links. We evaluate 
the following two schemes: 

• Scheme-1: Both S-fiows and M-flows are considered to be 
short-lived flows. 

• Scheme-2: An M-flow is considered to be long-lived 
before its last packet arrives, and to be short-lived after that. 
The performance of these two schemes are shown in Figure 

[T] where WS with Uniform Tie-breaking Rule is used as the 
scheduling algorithm. We can see that the performances are 
substantially different (note that the network is stable under 
both schemes). The number of queued bits of M-flows under 
Scheme-1 is larger than that under Scheme-2 by two orders of 
magnitude. This is because even an M-flow contains a huge 
number of bits (10, 000 on average), it can be served only 
when the link rate is 50 under Scheme-1. This simulation 
suggests that when the performance we are interested is at 
a small scale (e.g. acceptable queue-length being less than or 
equal to 100) compared with the size of the flow (e.g., 10 4 
in this simulation), the flow should be viewed as a long-lived 
flow for performance purpose. 

Simulation II: The Impact of Learning Period D 

In this simulation, we investigate the impact of D on the 
performance of WSLU. Recall that it is nature to choose 
D = oo for purely throughput-optimality considerations, but 
the disadvantage is that a flow may stay in the network for 
a very long time if the best link state occurs very rarely. 
We consider a network consisting of S-fiows, which arrive 
according to a truncated Poisson process with maximum value 
100 and mean A, and three L-flows. All links are assumed to 
be R-links. Figure |2] depicts the mean and standard deviation 
of the file-transfer delays with D = 16 and D = oo when the 
traffic load is light or medium. As we expected, the standard 
deviation under WSLU with D = oo is significantly larger 
than that under WSLU with D = 16 when A is large. This 
occurs because the best link rate 100 occurs with a probability 
0.01. This simulation confirms that in practical systems, we 
may want to choose a finite D to get desired performance. 



7 















-•- Scheme- 1 








-▼-Scheme-2 









































































0.04 0.06 0.08 
Arrival rate of S-flows (X) 




0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 
Arrival rate of S-flows (X) 



Fig. 1. Scheme-1 treats M-flows as short-lived flows, and Scheme-2 treats 
M-flows as long-lived flows 
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Fig. 2. The performance of WSLU with D 
traffic load is light or medium 



16 and D = oo when the 



Further we would like to comment that while the WSLU 
algorithm with a small D has a better performance in light 
or medium traffic regimes, throughput optimality is only 
guaranteed when D is sufficiently large. Figure [3]illustrates the 
average number of S-flows and average file-transfer delay for 
D = 16 and D = oo in heavy traffic regime. We can observe 
that in the heavy traffic regime, the WSLU with D = oo still 



stabilizes the network but the algorithm with D — 16 does not. 
So there is a clear tradeoff in choosing D: A small D reduces 
the file-transfer delay in light or medium traffic regimes, but 
a large D guarantees stability in heavy traffic regime. 
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Fig. 3. The performance of WSLU with D 
traffic load is heavy 
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Simulation III: Performance comparison of various algorithms 

In the following simulations, we choose D = 16. In the 
introduction, we have pointed out that the MaxWeight is not 
throughput optimal under flow-level dynamics because the 
backlog of a short-lived queue does not build up even when 
it has not been served for a while. To overcome this, one 
could try to use the delay of the head-of-line packet, instead of 
queue-length, as the weight because the head-of-line delay will 
keep increasing if no service is received. In the case of long- 
lived flows only, this algorithm is known to be throughput- 
optimal 0. We will show that this Delay-based scheduling 
does not solve the instability problem when there are short- 
lived flows. 

Delay-based Scheduling: At each time slot, the base station 
selects a flow i such that i E argmaxi Di(t)Ri(t), where 
Di(t) is the delay experienced so far by the head-of-line packet 
of flow i. 

We first consider the case where all flows are S-flows, which 
arrive according to a truncated Poisson process with maximum 
value 100 and mean A. An S-flow is assigned with a G-link 
or a P-link equally likely. 

Figure |4] shows the average file-transfer delay and average 
number of S-flows under different values of A. We can see 
that WSLU performs significantly better than the MaxWeight 
and Delay-based algorithms. Specifically, under MaxWeight 
and Delay-based algorithms, both the number of S-flows and 



8 



file-transfer delay explode when A > 0.102. WSLU, on the 
other hand, performs well even when A = 0.12. 
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Fig. 5. The performance of the Delay-based, MaxWeight, and WSLU 
algorithms in a network with both S-flows and L-flows 



Fig. 4. The performance of the Delay-based, 
algorithms in a network without L-flows 



MaxWeight, and WSLU 



for the practical scenario when the base station limits the 
number of flows in the network. 



Next, we consider the same scenario with three L-flows in 
the network. Two of the L-flows have G-links and one has 
a P-link. Figure shows the average number of short-lived 
flows and average file-transfer delay under different values of 
A. We can see that the MaxWeight becomes unstable even 
when the arrival rate of S-flows is very small. This is because 
the MaxWeight stops serving S-flows when the backlogs of 
L-flows are large, so S-flows stay in the network forever. The 
delay-based scheduling performs better than the MaxWeight, 
but significantly worse than WSLU. 

Simulation IV: Blocking probability of various algorithms 

While our theory assumes that the number of flows in 
the network can be infinite, in reality, base stations limit the 
number of simultaneously active flows, and reject new flows 
when the number of existing flows above some threshold. In 
this simulation, we assume that the base station can support 
at most 20 S-flows. A new S-flow will be blocked if 20 S- 
flows are already in the network. In this setting, the number 
of flows in the network is finite, so we compute the blocking 
probability, i.e., the fraction of S-flows rejected by the base 
station. 

We consider the case where no long-lived flow is in the 
network and the case where both short-lived and long-lived 
flows are present in the network. The flows and channels are 
selected as in Simulation III. The results are shown in Figure [6] 
and [7] We can see that the blocking probability under WSLU 
is substantially smaller than that under the MaxWeight or the 
delay-based scheduling. Thus, this simulation demonstrates 
that instability under the assumption when the number of flows 
is allowed to unbounded implies high blocking probabilities 
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Fig. 6. The blocking probabilities of the Delay-based, MaxWeight, and 
WSLU in a network without L-flows 
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Fig. 7. The blocking probabilities of the Delay-based, MaxWeight, and 
WSLU in a network with L-flows 



Simulation V: WSLU versus WSLO 

In this simulation, we study the impact of tie-breaking rules 
on performance. We compare the performance of the WSLU 
and WSLO. We first study the case where the base station 
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does not limit the number of simultaneously active flows and 
there is no long-lived flow in the network. The simulation 
setting is the same as that in Simulation III. Figure [8] shows 
the average file-transfer delay and average number of S-flows 
under different values of A. We can see that the WSLO reduces 
the file-transfer delay and number of S-flows by nearly 75% 
when A = 0.13, which indicates the importance of selecting a 
good tie-breaking rule for improving the network performance. 
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Fig. 8. The performance of the WSLU and WSLO algorithms in a network 
without L-fiows 



Next, we study the case where the base station does not limit 
the number of simultaneously active flows and there are three 
L- flows in the network. Figure [9] shows the average number of 
short-lived flows and average file-transfer delay under different 
values of A. We can see again that the WSLO algorithm has 
a much better performance than the WSLU, especially when 
A is large. 

Finally we consider the situation where the base station can 
support at most 20 S-flows. A new S-flow will be blocked if 
20 S-flows are already in the network. The simulation setting 
is the same as that in Simulation IV. We calculate the blocking 
probabilities, and the results are shown in Figure [10] and QT| 
We can see that the blocking probability under the WSLO is 
much smaller than that under the WSLU policy when A is 
large. 

VII. Conclusions and Discussions 

In this paper, we studied multiuser scheduling in networks 
with flow-level dynamics. We first obtained necessary condi- 
tions for flow-level stability of networks with both long-lived 
flows and short-lived flows. Then based on an optimization 
framework, we proposed the workload-based scheduling with 
learning that is throughput-optimal under flow-level dynamics 
and requires no prior knowledge about channels and traffic. In 
the simulations, we evaluated the performance of the proposed 
scheduling algorithms, and demonstrated that the proposed 
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Fig. 9. The performance of the WSLU and WSLO algorithms in a network 
with both S-flows and L-flows 
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Fig. 10. The blocking probabilities of the WSLU and WSLO in a network 
without L-flows 
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Fig. 11, The blocking probabilities of the WSLU and WSLO in a network 
with L-flows 



algorithm performs significantly better than the MaxWeight 
algorithm and the Delay-based algorithm in various settings. 
Next we discuss the limitations of our model and possible 
extensions. 
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A. The choice of D 

According to Theorem [3] the learning period D should 
be sufficiently large to guarantee throughput-optimality. Our 
simulation results on the other hand suggested that a small 
D may result in better performance. Therefore, there is clear 
trade-off in choosing D. The study of the choice for D is one 
potential future work. 



B. Unbounded file arrivals and file sizes 

One limitation of our model is that the random variables 
associated with the number of file arrivals and file sizes are 
assumed to be upper bounded. One interesting future research 
problem is to extend the results to unbounded number of file 
arrivals and file sizes. 



Appendix A: Proof of Theorem|2] 



ei(t) 



We define i?jf ax to 



Recall that W s (t) = £ ; 
be the largest achievable link rate of class-/s short-lived flows 

and A s (t) = E M E 



A k (t) 



fi 



which is the amount of 



new workload (from short-lived flows) injected in the network 
at time t, and fi s (t) to be the decrease of the workload at time 
t, i.e., n s (t) = 1 if the workload of short-lived flows is reduced 
by one and /i s (<) = otherwise. Based on the notations above, 
the evolution of short-lived flows can be described as: 

W,(t+l) = W,(t)+A a (t)-n a (t). 
Further, the evolution of Qi(t) can be described as 

Qi{t + 1) = Qi{t) + Xi(t) - m(t) + ui(t), 

where ni(t) is the decrease of Qi(t) due to the service long- 
lived flow I receives at time t, and is the unused service 
due to the lack of data in the queue. 

We consider the following Lyapunov function 



V(t)=a(W s (t)) 2 + Y,mt)f 



(8) 



lec 



We will prove that the drift of the Lyapunov function satisfies 

E[V(f + 1) - V(*)|M(f)] < U d l mt)er - | [aXW 8 (t) 



lec 



M(t)£T 



for some Ud > 0, A > and a finite set T (the values of 
these parameters will be defined in the following analysis). 
Positive recurrence of M then follows from Foster's Criterion 
for Markov chains lfl2l . 

First, since the number of arrivals, the sizes of short-lived 
flows and channel rates are all bounded, it can be verified that 



there exists U, independent of M(t), such that 
E[V(t + l)-V{t)\M{t)} 
=E [a (W s (t + l)) 2 - a (W s (t)) 2 + 



M(t) 



lec lec 
<U + 2aW s (t)E [A s (t) - fi s (t)\ M(t)} + 

2j2QimiXt(t)-t*i(t)\M(t)} 




£A fe E 



keic 



-E[/x s (*)|M(t)] 

+ 2Y / Qi(t) (x,-E[ W (t)|M(t)]). 
lec 

Recall that we assume that (1 + e)xi and (1 + e)Xk satisfy 
the supportability conditions of Theorem 03 By adding and 
subtracting corresponding p c ,iR c ,i and /i c ,s, we obtain that 

E[V(t + l) -V(t)\M(t)] -U 

< 2aW s (t)E [E [/x c , s - (i s (t)\ C(t) = c]\M{t)} 

+2^Qi(m [E [p c ,iR c ,i - C(t) - c]|M(t)] 



lec 



-2eaW s {t)\-2€^Qi{t) 



lec 



where 



a = hr A fc E 



\keK 



•'■/. 



Next we assume C(t) = c and analyze the following 
quantity 

aW s (t) - ft s (t)) + Ol(*) (Pc,iR c j - w( f )) • ( 9 ) 
lec 

We have the following facts: 

• Fact 1: Assume that there exists a short-lived flow i such 
that R t (t) = i?f lax or R^t) > Q t (t). If a short-lived 
flow is selected to be served, then the workload of the 
selected flow is reduced by one and = 1. If long- 
lived flow I is selected, the rate flow I receives is R c ,i- 
Thus, we have that 

aW s (t)fi s (t) + J2Ql( t )M(t) 
lec 

= max{aW s (<), max z Qi(t)R c j} 

> aW s {t)fi CiS + Y Qi(t)p c ,iRcd, 
lec 

where the last inequality holds because J2i Pc.i + A*c,s < 
1. Therefore, we have ^ < in this case. 

• Fact 2: Assume that there does not exist a short-lived 
flow i such that R t {t) = J?f ax or R % (t) > Q t {t). In this 
case, we have 

© < aW s (t)+maxQi(t)R Ct i 

< aW s (t)+ i? max max QAi). 

lec 
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Now we define a set T such that 

T = {M : W s < U w and Q t < U Q VZ} , 
where Uw is a positive integer satisfying that 

(1 _ < | min { A, mi °' I g4 Xi } = ei (10) 

(ID 



y — eaX 1 



and Uq is a positive integer satisfying 

> ^±R. (12) 

emin /e£ a:; 

We next compute the drift of the Lyapunov function accord- 
ing to the value of M(t). 

• Case I: Assume M(t) e T. According to the definition 
of T, we have 

E[V(t + 1) - V{t)\M{t)} <U + 2aU w + 2i? max L£/ Q . 

• Case II: Assume W s (t) > Uw . Since the size of a short- 
lived flow is upper bounded by F max , W s (t) > Uw 

short-lived flows are in the 



u w 



implies that at least 
network at time t. Define S(t) to be the following 
event: no short-lived flow satisfies Ri(t) = i?f lax or 
Ri(t) > Qi(t) . 
Recall that 

minPr(i^(t) = i?,™ ax ) > p™ ax . 

i 

Given at least jj^ x short-lived flows are in the network, 
we have that 

Pr(l 5(t) = 1)<(1- P 1 ™)-"" 



< ei 



According to facts 1 and 2, © is positive only if S(t) 
occurs and the value of © is bounded by aW s (t) + 
i? max max; e £ Qi(t). Therefore, we can conclude that in 
this case (Case II), 



E[V(t+l)-V(t)\M(t)} 



< 



i? max max Qi(t) 



U + 2ei \ aW s {t) 
-2eaW s {t)\-2e^Qi{t)xi 

< U - eaW s (t)\ - eJ2Ql(t>i 
lec 



aXW s (t) 



lec 



Qi(t)xi 



(13) 



(14) 



where inequality (13[ holds due to the definition of ei 
( [Tol l, and inequality < fl~4T > holds due to inequality ( fTTT ). 
Case III: Assume that W s (t) < Uw and Qiit) > Uq 
for some I. In this case, if a long-lived flow is selected 
for a given c, we have 

® < aW s (t)^ s < aW s (t). 

Otherwise, if a short-lived flow is selected, it means for 
the given c, we have max; Qi(t)R Ci i < aW s (t), and 

© < 2aW s (t). 



□ Therefore, we can conclude that in this case, 

E[V(t+ 1) - V(t)\M{t)] 

<U + iaW s (t)-2eaW s (t)\-2eY^Qi{t)xi (15) 

lec 

<U + 4aU w -2eaW s (t)\-2e^2Qi(t)xi 



lec 



e 

< 

- 2 



a\W s (t) +J2Qi(t) 



xi 



lec 



(16) 



where the last inequality yields from the definition of Uq 
01. 

From the analysis above, we can conclude that 



E[V(t+l)-V(t)\M(t)] <U d l m 



(t)GT 



[a\W a {t) 



■n 



lec 



M(t)£T7 



where U d = U + 2aU w + 2R max LU Q and T is a set with 
a finite number of elements. Since V(t) > for all t, the 
Lyapunov function is always lower bounded. Further the drift 
of the Lyapunov is upper bounded when M(£) belongs to a 
finite set T, and is negative otherwise. So invoking Foster's 
criterion, the Markov chain M(t) is positive recurrent and the 
boundedness of the first moment follows from lfl3l . 

Appendix B: Proof of Theorem[3] 

Consider the network that is operated under WSL, and 
define H(t) to be 

Now given "H(t), we define the following notations: 

> Define /J,2-.i(t) = Ri(t) if flow I is selected by WSL, and 
H2-,i{t) = otherwise. 

> Define /i2:i(t) = 1 if flow i is selected by WSL and the 
workload of flow i can be reduced by one, and fi2-i(t) = 
otherwise. 

• Define (ii-i(t) = Ri(t) if flow I is selected by WS, and 
A*i;/(i) = otherwise. 

• Define fJ,i ; i(t) = 1 if flow i is selected by WS and the 
workload of flow i can be reduced by one, and = 
otherwise. 

We remark that ^-j {t) is the action selected by the base station 
at time t under WSL and fii-j(t) is the action selected by the 
base station at time t under WS, assuming the same history 
H{t). 

We define the Lyapunov function to be 

V(n) =a(W s (nT)f +Y,(Qi(nT)) 2 . (17) 

lec 

This Lyapunov function is similar to the one used in the proof 
of Theorem |2] and we will show that this is a valid Lyapunov 
function for the workload-based scheduling with learning. 
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Then, it is easy to verify that there exists U\ independent of 
M(n) such that 

E[V(n + l) -V{n)\M{n)} 

(n+l)T-l 

t=nT 

(n+l)T-l 

^2E Q,(nT) £ (X,(i)- M2 ;i(*)) M(n) 



lec 



We define 

Diff(t) =aW,(t) {^(t) - fx 2 . s (t)) 



and 



t=nT 



w s (t)= £ 

tei(t) 



Dividing the time into two segments [nT, nT + D — 1] and 
[nT + D, (n + 1)T - 1], we obtain 

E[V(n + l) - V(n)\M(n)] 
<Ux + 2aW s {nT)\D + 2^2Qt{nT)xiD 

leC 
(n+l)T-l 

\W s {nT) J2 (Mt)-»2-At)) 



E 2E 

lec 



t=nT+D 
(n+l)T-l 



M(n) 



t=nT+D 



M(n) 



Next, we compute its value in three different situations: 

• Situ-A: Consider the situation in which aW s (t) < 
maxi e cQi(t)Ri(t). We note that W s (t) > W s (t) since 

< i?™ ax for all t and i. Therefore, given 

aW s {t) < J^iecQii 1 )' both WS and WSL wil1 select 
a long-lived flow. In this case, we can conclude that 

fii;l(t) = (JL2-,i(t) and /ii ;s (t) = H2;s(t) = 0, 

and Diff(i) = 0. 

• Situ-B: Consider the situation in which aW s (t) > 
max i6 £ Qi(t)Ri(t). In this case, both WS and WSL will 
select a short-lived flow, which implies that 



Note that \Qi(h) - Qi(t 2 )\ and \W k (h) -Wk(t 2 )\ are both 
bounded by some constants independent of M(n), so there 
exists U such that 

E[V(n + l) - V(n)\M(n)} 
<U + 2aW s (nT)XD + 2 ^ Qi{nT) Xl D 



2E 



ze£ 

(n+l)T-l 



M(n) 
M(n 



"(n+X)T-l 

+ ^2E E QK*)W(*)-/*a ; i(*)) 

;e£ t=nT+D 

Now, by adding and subtracting /ii ; .(t), we obtain 
E[^(n + l)-y(n)|M(n)] 

(n+l)T-l 

<C/ + 2aV^(nT)AL> + 2E ( 3i(" r ) a; i- D + E Drift (*). 



t=nT+D 



where 



Drift(t) 



=2E 



M(n) 



(18) 



aW.(t)A.(t) + 53Qi(*)^i(*) 
ze£ 

- 2E[aW,(t)Ati ; .(t) + E Ql(*)W:i(*)|M(n)] (19) 
ze£ 

+ £ 2E[Q,(t) (A*i ; i(t) - M2;i(*)) |M(n)] (20) 



lec 
+ 2E 



[aW a (f) (/*!.,(*)- W; .(*))|M(r 



(21) 



Note that (|20]l+(|2T|i is the difference between WS and WSL. 
In the following analysis, we will prove that this difference is 
small compared to the absolute value of (IT8Ti+(|T9T>. 



and 



Hl;l(t) = »2;l(t) = 0, 

Diff(i) =aW s {t) (/x 1;s (t) - fi 2 ;s{t)) 

< aW s (t) (1 - fl2;s(t)) ■ 

• Situ-C: Consider the situation in which aW s (t) > 
max.i e cQi{t)Ri(t) > aW s (t). In this case, WS will 
select a long-lived flow and WSL will select a short-lived 
flow. We hence have 



and 



Hi-,i{t) > and ^(t) = fJ- 2 -i(t) = 0, 



Diff(i) = maxQt(t)Ri(t) - aW s (t)fi 2 , s {t) 
< aW s {t) -aW s (t)fi 2 , s (t) 



According to the analysis above, we have that 
E[Diff(i)|M(n)] 



□ 



<E 



i;W s (t)|Situ-B,/i 2i . s = 0,M(r 



Pr Situ-B, ix 2 -s = 0|M(n) 



E 



aW^ s (i)|Situ-C,^ 2;s =0,M(n) 



Pr ( Situ-C, fi 2 - s = 0|M(n) 



E 



aW s (t) -aW fl (t)|Situ-C,At 2ia = l,M(n) 



Pr Situ-C, H2-s = l|M(?i) 



Next we define a finite set T. We first introduce some 
constants: 



ei — mm 



J Ae e min; x L I 
\ 32 ' 8fl max J 
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£2 



32R" 



and D fo and N e ~ are the numbers that 



guarantee Pi (£ miss (t)) < e 2 , which are defined by the 
goodness of the tie-breaking rule. 
. A^ ax = K A max i^ max ; which is the maximum number 
of bits of short-lived flows injected in one time slot, and 
also the upper bound on the new workload injected in the 
network in one time slot. 

We define a set T such that 



T - I M(n) ■ W s (nT)<U w+2 T+°-I^f^ 



In this definition, Uw is a positive integer satisfying that 



(1 -pf^)T^ < £l , 



(22) 



U 



+ 16e2QA?;, ax T+8aDi? n 



'+16£ 2 afl max T+8A^. ax ^ 23 ^ 



> N r . 



and Uq is a positive integer satisfying 

2 Ei »i" m °? 



8(7+12afl mi * x (LV + - 



+ (A™ ax +2)T) 



(24) 



(25) 



Since the changes of W s (t) and Qi(t) during each time slot is 
bounded by some constants independent of M(n), it is easy 
to verify that T is a set of a finite number of elements. 

Next, we analyze the drift of Lyapunov function case by 
case assuming that 



D > 



log Ae — log 16 — log R 1] 
log(l-pf ax ) 



(26) 



and T > 



(<i+e)D 



• Case I: Assume that M(n) £ T. In this case, it is easy to 
verify that E[V(n + 1) — V(n)|M(Ti)] is bounded by some 
constant Ud- 

• Case II: Assume that 



W s {nT) >U W + 2T 



2Y,iXiR max T 



>U W + T. 



Recall that £ m i ss (t) is the event such that the tie-breaking 
rule selects a short-lived flow with Rf 111 *^) ^ i?™ ax . Note 
that /i2 ; s(i) = implies that £ m i SS (t) occurs. Also note the 
following facts: 

- For any nT < t < (n + 1)T, we have W(t) < 
W{nT) + A$? X T, 

- Given W s (nT) > U w + T, we have W s (t) > U w 
for all nT < t < (n + 1)T - 1. Then according to 
the definition of e 2 and Uw and assumption that the 
tie-breaking rule is good, we have 

Pr(£ mlss (t)) < e 2 

for all nT + D < t < (n + 1)T - 1. 



Given any M(n) and any nT + D < t < (n+ l)T - 1, 
we have 



E 



aW s {t) - aW s {t)\Situ-C, = 1,M(») 



Pr (situ-C, u 2 - s = l|M(n) 



<E 
=E 
<E 

<E 



aW s {t) -aW s (t)\M(n) 



E 



W s (t) -aW s (t) W s (t-D)\ M(») 



a{l-p^ ax ) u W s {t-D)R D 



aA^ ax L>|M(n) 
(27)" 



a(l - P T x ) D (W s (t) + L>)i? max + a\^D\M(n) 



where the inequality d27l i holds because at most A^ ax D 
bits belonging to short-lived flows are in the network for 
less than D time slots at time t, and a flow having been 
in the network for at least D time slots can estimate 
correctly its workload with a probability at least 1 — 

(i-pD d 

Now according to the observations above, we can obtain 
that 

E[Diff(i)|M(n)] 

<e 2 a (W s (nT) + A^ ax T) + e 2 a (R™*W s (nT) + A^ ax T) 



E 



a(l - vT*) D {W s {t) + D)i? max + aA^ ax ^|M(? 



Combining with the analysis leading to ( TT3b in Appendix 
A, we conclude that 

Drift(i) 



<2E 



ei ( aW s (t) + i? max max Qi (t) 



eaW,(jt)X-e^2Q l {t) 



■>1 



lec 



+ e 2 a(W s (nT)+X^T) 

+ e 2 a {R™ K W s {nT) + A^ ax T) 

+ a(l -jn^f.ft) + L>)i? max + aA^ ax £>|M(n) 



<E 



-e (aXW s (t) +Y^xiQi{t) 



M(n) 



where the last inequality holds due to (f23b . 
> Case III: Assume that 



W s (nT) <U W + 2T 



2 Ez xiR max T 
aX 



and 



mTKff , 2aAT 2TJ? max E ; ^ 

Qi(nT) >Un H : 1 ; 

mm; x/ mm; xi 

for some /. In this case, we have 

Diff(i) < aW s (t) < aR max W s (t). 



> U C 
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Combining with the analysis leading to (fT3T > in Appendix 
A, we have that 

Drift(t) 

<2E [aR ma *W s (t) + 2aW s (t) 



<E 



e(a~XW s (t)+Y,xiQi(t) 
-e la\W s (t)+J2xiQi{t) 



tec 




where the last inequality holds due to 

Now, combining case II and case III, we can obtain that 

E[V(n + l) -V(n)\M(n)] 
<U + 2aW s {nT)XD + 2 ^ Qi(nT)x t D 



□ 



lec 



(n+l)T-l 



t=nT+D 



-e laXW s (t) + ^2xiQi(t) 

<{/ + 2aW s (nT)A£» + 2 ^ Qi(nT)x t D 
lec 

- e(T — D) la\W s {nT) +^xiQi{nT)\ 

\ IGC ) 

+ e(T - £>)(aAT + ^ Xl R max T) 



M(n) 



(n+l)T-l 

<-C7- ]T E 

t=nT+D 



| (aAW 8 (t)+53aj,Q,(*) 



M(n) 



where the last inequality yields from the definition of Uw and 
Uq. Finally, we can conclude the theorem from lfl2l . lf]~3l . 

Appendix C: The Uniform Tie-breaking Rule 

Recall that we define £ m i SS (t) to be the event that the tie- 
breaking rule selects a short-lived flow with i?f ax (i) ^ Rf 1 ax . 
Proposition 4: The uniform tie-breaking rule is good. 
Proof: Suppose set 

J(t) = : Ri(t) = Rr x (t) or R t (t) > Q 4 (t)} . 

Under the uniform tie-breaking, £ m i SS (t) occurs with proba- 
bility 



i:iej(t) and i?f lax (t) # i?? lax j 



< 



|i : Rf la *(t) ^ i?f lax j 



{» : Ri(t) = Hr*(t)} 



Assume that N short-lived flows are in the network at time 
t — D and denote by M the set of these short-lived flows. Our 
proof contains the following two steps: 

Step 1: We first obtain an upper bound on 



{i : R? ax (i) 7^ i?f lax and i e Afj 



Considering a short-lived flow (flow i) which is in the network 
at time t — D, we have 



^ < (i - pT x ) d = 



Thus, E[iVi] < eN. According to the Chernoff bound, we have 



< exp 



< exp 



Pr(A^i > l.UN + D) 

{l.leN + D - ElN^) 2 



3E[iVi 
(O.leN + D) 2 



3eN 

< exp (-0.003eA^ - OMD) . 

Next note that at most KX msK short-lived flows join the 
network during each time slot, so we can conclude that 



Pr ( ji : Rf™*(t) ^ R 



IX 1 



> l.UN + KX m **D + D 



< exp (-0.003eA - 0.06D) . 

Step 2: Since at most one flow can be completely transmit- 
ted in one time slot, so least N — D flows are in the network 
at time t, each having a probability at least p™ ax to be in the 
best channel state. 



< 0.9p™ ax (A - D) 



Pr (\{i:R i (t) = R?°*(t)} 

< Pr (|{» : Ri(t) = i?r x (*)}l < 0.9p™ aK (N - D)) 

< exp (-0.003^ iax (Af -£>)). 



Summary: From step 1 and step 2, we can conclude that 

I 



< 



Pr(^ m j SS (£)) 
l.UN + KX" 



cxp(-0.0Q3p™ x (N - D)) 



0.9pf^(N - D) 

+ exp (-0.003eA- 0.061?), 

which converges to zero as both D and N/D go to infinity. 
The proposition holds because the sizes of short-lived flows 
are bounded and a large workload implies a large number of 
short-lived flows. ■ 



Appendix D: Oldest-first Tie-breaking Rule 

Proposition 5: The oldest-first tie-breaking is a good tie- 
breaking rule. 

Proof: We assume that at time slot t — D, there are N > 
2D 2 short-lived flows in the network. We group short-lived 
flows into groups Q according to the time they arrived at the 
network such that group Gf(t) contains all flows arriving no 
less than f time slots ago at time t, and group Q T (t) contains 
the flows arriving exact r time slots ago at time t (r < f ). 

Case 1: Assume that \Q f {t — D)\> D 2 . We first consider 
the following probability 

Pr (a flow £ Gf(t) is selected at t and R™ ax (t) ^ Rf 1 ^ . 

Note that Qfif) can contain at most KX maK D additional 
flows compared to Q f (t — D) since \Q T \ < KX niax for all 
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t < t. Following the analysis for the uniform tie-breaking in 
Appendix C, we can easily prove that 



Pr fa flow e g t (t) is selected at t and R™ ax (t) / 




as D goes to infinity. 

Next, note that at most one short-lived flow can be com- 
pletely transmitted in one time slot, so Qf(t) containing at 
least D 2 — D flows at time t, which implies that 

c (t) ^ R ma A 



Pr [a flow Q f (t) is selected at t and Rf a ' 

<(i~p^ x ) d2 - d . 

Therefore, we conclude that 

the selected flow at t has R^ ax (t) ^ 



= Pr [a flow Q f {t) is selected at t and R™ ax (t) ^ Rf ax j 
+ Pr fa flow g g f (t) is selected at t and flf ax (i) ^ i?. max ^ 



J 

which converges to zero as D goes to infinity. 

Case 2: Assume that \Q T (t — D)\ < D 2 . In this case, we 
search the groups starting from group Q f (t) and stop at group 
t* if D 2 + DKX max > Yfr=r- \Sr(t)\ > D 2 . Note that when 
D is sufficiently large, such r* exists and r* > D because 
N > 2D 2 and \Q T \ < A'A max for all r < f. Considering a 
certain flow i such that i G U^ =r *^ T (t), we have that 

Pr (flow i is selected at t and ^"({J ^ i?" ax ) 



< Pr ax (t) ^ i? 4 max 

max\D A 



[41 



[51 



[61 



[71 



[81 



[9] 



[10] 
[11] 



[12] 



[13] 



< (i - Ps r = e- 

Thus, we can obtain that 

Pr (a flow e \f T=T ,Q T {t) is selected at t and ^ nax (t) ^ i?" ax ) 
<(L» 2 +MA max )e, 

which converges to zero as D goes to infinity. Further, similar 
to the analysis in Case 1, we can obtain that when D is 
sufficiently large, 

Pr (a flow g lCo 1 ^) is selected at t and ^"(i) ^ i?' nax ) 

which converges to zero as well. 

Combining Case 1 and 2, we can conclude that the oldest- 
first tie-breaking is a good tie-breaking rule. ■ 
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